The present invention generally relates to the technical field of face recognition. More particularly, the invention involves the creation of facial features, termed “face pattern words” and “face pattern bytes”, used for face identification. The invention provides a means for identifying individuals based on visible and/or thermal images of those individuals by utilizing software systems and methods implemented by instructions on a computer system and a computer readable medium containing instructions on a computer system for face identification. The invention and its optimal orientation bit code and optimized binary coding can also be used for other pattern recognitions such as fingerprint recognition, iris recognition, and the like.
Many challenges exist for face recognition using currently-available techniques. Compared to iris or fingerprint recognition, current face recognition technology has relatively low identification rate and slow recognition speed due to high algorithm complexity and large image size. However, methods using iris or fingerprint image acquisition are time-consuming and require the subject's collaboration. The present invention provides a novel system and methodology of face identification using face pattern words and face pattern bytes.
Many algorithms have been developed in the face recognition domain over the past two decades. Three common algorithms (according to NIST) are Principle Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Elastic Bunch Graph Matching (EBGM) and are briefly summarized as follows. PCA is used for dimensionality reduction to find the eigenvectors which best account for the distribution of all face images in a database. These eigenvectors are commonly referred to as eigenfaces. The number of eigenfaces depends on a particular implementation and the database images. For example, the number of eigenfaces may be up to 100. All faces, including the training images (in the database) and the probe image, are projected onto eigenfaces to find a set of weights (feature vector) for each face. Each weight in a set actually describes the contribution of the corresponding eigenface. Identification of a face image means to find the closest face (i.e., feature vector) in the database according to a distance measure. A PCA algorithm typically requires frontal face images and all images should have the same size and line up the eyes and mouth of the subjects. Unlike PCA, LDA is used to find an efficient way to represent the faces using the face class information. The Fisherface algorithm is derived from the Fisher Linear Discriminant, which uses class specific information. By defining different classes with different statistics, the images in the training set are divided into the corresponding classes. Then techniques similar to the PCA algorithm are applied. The Fisherface algorithm results in a higher accuracy rate in recognizing faces when compared with the Eigenface algorithm. In the EBGM algorithm, faces are represented as graphs, with nodes positioned at fiducial points (such as the eyes, tip of nose, and mouth), and edges are labeled with 2-D distance vectors. Each node contains a set of 40 complex Gabor wavelet coefficients, including both phase and magnitude, known as a jet. Wavelet coefficients are extracted using a family of Gabor kernels with 5 different spatial frequencies and 8 orientations. A graph is labeled using a set of nodes connected by edges, where nodes are labeled with jets and edges are labeled with distances. Thus, the geometry of a face is encoded by the edges whereas the gray value distribution is patch-wise encoded by the nodes (jets). To identify a probe face (also called query face), the face graph is positioned on the face image using elastic bunch graph matching, which actually maximizes the graph similarity function. After the graph has been positioned on the probe face, it can be identified by comparing the similarity between the probe face and each face stored in the database (referred to as gallery face thereafter). From the literature reports regarding the comparisons of these three common algorithms, it seems that the EBGM algorithm is the best in terms of identification rate and performance reliability.
The performance of face recognition using visible imagery depends much on the illumination conditions, which can dramatically reduce performance with poor illumination, especially at nighttime. To address the limitations of using visible face images, recently researchers have begun investigating face recognition approaches using thermal (long wave infrared, LWIR) images or the fusion of both thermal and visible images. Buddharaju et al. proposed to localize the superficial vessel network using thermal images and then to extract the branching points (referred to as Thermal Minutia Points) from the vascular network as features for face recognition. Similar work was reported by Akhloufi and Bendada, where blood vessels were used as a thermal faceprint. Bebis et al. suggested two fusion schemes for eigenface recognition, i.e., pixel-based and feature-based fusion of visible and thermal imagery. Arandjelovic et al. proposed a decision level fusion of visual and thermal images. Both experimental results showed improvement using fusion and revealed a performance decrease when eyeglasses were present in the image. However, thermal face recognition has not been sufficiently investigated, especially in the fields of face pattern representation and recognition algorithms.
As mentioned, face recognition techniques (like PCA, LDA, and EBGM) using visible images are very much restricted by the illumination conditions. The face recognition system and method of present invention was originally developed on thermal face images and later extended to visible images. Although other face recognition methods exist (like thermal faceprint) using thermal images, those methods have not been sufficiently tested on a large database and none have been tested using both visible and thermal images. The face recognition system and method of the present invention can identify a subject using either a thermal image (both daytime and nighttime) or using a visible image (daytime), which can be applied to any field 24 hours a day regardless of illumination conditions.
U.S. Pat. No. 6,563,950 discloses an image analysis method called “labeled bunch graph” that can be used for face recognition called elastic bunch graph match or EBGM. Gabor wavelet transforms (GWT) with 5 frequencies and 8 orientations are applied to each face image. A similarity metric between two face images is defined with a number of GWT magnitudes that are extracted from the predefined nodes on each face image. A probe face is identified by the gallery face that has the largest similarity value (i.e., the most similar to the probe face). However in the present invention, GWT coefficients are used for facial feature extraction but the GWT coefficients with 8 frequencies and 16 orientations are applied to each face image. Moreover for the present invention, the index of the maximal GWT magnitude among 16 orientations at each frequency band is extracted then encoded as a 4-bit orientation bit code. Eight (8) sets of 4-bit orientation bit code are put into a 32-bit face pattern word (pixel by pixel) and the 4-bit orientation bit code is optimized with respect to Hamming Distance (HD). A probe face is identified by the gallery face that has the smallest HD value.
U.S. Pat. No. 7,860,280 describes a face detection method by locating the facial features (i.e., eyes, nose, mouth) from a color face image (visible image). However, no specific face recognition technique is provided in that technology.
U.S. Pat. No. 7,873,189 discloses a face recognition method using a support vector machine (SVM) classifier. A face image is divided into three sections in horizontal and vertical directions, respectively. Then PCA (principle component analysis) and LDA (linear discriminant analysis) are applied to the six divided images, the resulted feature vector of each divided image is compared with the feature vector stored in a database in advance to calculate a similarity value. A new feature vector is formed with the six similarity values and is used to train a SVM classifier to face recognition. This classifier-based (SVM) face recognition method is completely different from the present invention which utilizes Gabor wavelet transforms for feature extraction and uses Hamming Distance for face matching.
U.S. Pat. No. 7,876,931 describes a face recognition method that models a 3D face using a 3D camera, then derives a number of 2D face images from the 3D face model and enrolls the 2D images into a database. The derived 2D face images are used to match with the probe face images from a regular video camera. However, no specific face recognition technique is provided by this technology.
The face recognition system and method of the present invention uses Gabor wavelet transforms (GWT) to create face patterns (FPWs and FPBs), which is similar to the EBGM method. However, the EBGM method extracts the facial features from GWT coefficients at a number of fiducial points (about 31 points surrounding the eyes, tip of nose, and mouth). Locating the fiducial points on each face image is extremely time-consuming. The similarity function used in EBGM involves complicated float-number computations that slow down the face matching process. In the present invention, the face patterns (FPWs and FPBs) are simplified from GWT coefficients by using order statistics. The FPWs and FPBs comprise integer numbers. Furthermore, the Hamming Distance used for face matching mainly involves logical computations (AND, XOR), which expedite the face recognition process. It is also viable to implement a real-time face recognition system by using the present invention with proper hardware support.
Therefore, there exists a need for an accurate and efficient face recognition system and method using facial features for identification. The present invention provides such a system and method and software means for fast, accurate, and efficient face recognition.